Interactive Algorithms for Unsupervised Machine Learning

This thesis explores the power of interactivity in unsupervised machine learning problems. Interactive algorithms employ feedback driven measurements to mitigate the cost of data acquisition and consequently enable statistical analysis in otherwise intractable settings. Unsupervised learning methods are fundamental tools across a variety of domains, and interactive procedures promise to broaden the scope of statistical analysis. We develop interactive mechanisms and inference procedures for three unsupervised problems: subspace learning, clustering, and tree metric learning. Our theoretical and empirical analysis shows that interactivity can bring both statistical and computational improvements over non-interactive approaches. In addition, an over-arching thread of this thesis is that interactive learning is particularly powerful for non-uniform datasets, where non-uniformity is quantified differently in each setting. We first study the subspace learning problem, where the goal is to recover or approximate the principal subspace of a collection of partially observed data points. We propose statistically and computationally appealing interactive algorithms for both the matrix completion problem, where the data points lie in a low dimensional subspace, and the matrix approximation problem, where one must approximate the principal components of an arbitrary collection of points. We measure uniformity with the notion of incoherence, which is known to be necessary for non-interactive algorithms, and we show that our feedback-driven algorithms perform well under much milder incoherence assumptions. We next consider clustering a dataset represented by a partially observed similarity matrix. We propose an interactive procedure for recovering a hierarchical clustering from a small number of carefully selected similarity measurements. The algorithm exploits non-uniformity of cluster size by using few measurements to recover larger clusters and then focusing measurements on identifying the smaller structures. In addition to coming with strong statistical and computational guarantees, this algorithm performs well in practice. Finally we consider a specific metric learning problem, where we compute a latent tree metric to approximate distances over a point set. This problem is motivated by applications in network tomography, where the goal is to approximate the network structure using only measurements between pairs of end hosts. Our algorithms use an interactively chosen subset of the pairwise distances to learn the latent tree metric while being robust to either additive noise or a small number of arbitrarily corrupted distances. As before, we leverage non-uniformity inherent in the tree metric structure to achieve low sample complexity. Throughout we complement our theoretical results with empirical evaluations.